te-M« 


112  ESTIIMT1N6  SYSTEM  RELIABILITY:  MONTE  CMtLO  HETHOOS 
SENSITIVITY  RNO  ERRORS.  .  <U>  NORTH  CAROLINA  UNIV  AT 
CHAPEL  HILL  CURRICULUM  IN  OPERATIONS  R.  .  OS  FISHMAN 
UNCLASSIFIED  JAN  87  UNC70RSA7TR-8771  AFOSR-TR-87-1892  F70  1274 


171 


r  / 


OTIC  Flit  COPY  - 


AD-A186  182  afosr-tr.  8  7- 10 

I  OPERATIONS  RESEARCH  AND  SYSTEMS  ANALYSIS 


- -  ... 

— - 

Estimating  System 

Reliability: 

Monte  Carlo  Methods, 

Sensitivity  and 

Errors  in  Input 

Parameters 

George  S. 

Fishman 

Technical  Report  No. 

UNC/0RSA/TR-87/1 

January 

1  987 

UNIVERSITY  OF  NORTH  CAROLINA 
AT  CHAPEL  HILL 


DTTC 

^LECTEj 

^  OCT  0  7 1987 1 


•.IT  r.TlV.V’  ■«»»>»■»  imd  0ol«i  Hi 
la  anils!: 


-N.-Cv'\*A\v’.sVV-vV.V».v”lv.v.'»- 


8  7  9  2*4  2  80 


.  *"U  *v*v»\nr.i  irv  iruimnni 


0 


Estimating  System  Reliability: 
Monte  Carlo  Methods,  Sensitivity  and 
Errors  in  Input  Parameters 


George  S.  Fishman 


Technical  Report  No.  UNC/ORSA/TR-87/ 1 


January  1987 


klKSPECTEO 


Accession  for 

tfns  GRA&I 
D7IC  TAB 
Unannounced 
Justlf icatlon_ 


Tl 


By — - - 

Distribution/ 

Availability  Codes 
Avail  and/or 
Dlst  Special 


Curriculum  in  Operations  Research  and 
Systems  Analysis 

University  of  North  Carolina  at  Chapel  Hill 


DT!C 

ELECTEE^ 


OCT  0  7  1987 1  g 


Prepared  for  invited  presentation  at  the  Second  International 
Workshop  on  Applied  Mathematics  and  Performance  Reliability 
Models  of  C om pu t er / C omm un i ca t i on  Systems,  University  of  Rome, 
Italy,  May  25-29,  1987. 

This  research  was  supported  by  the  Air  Force  Office  of  Scientific 
Research  under  grant  A FO SR  -  8 4  -  01 M 0  .  Reproduction  in  whole  or 
part  is  permitted  for  any  purpose  of  the  United  States 
Government . 


1  a  ;J*. 


;  1  «., 


Abstract 


In  assessing  the  reliability  of  a  binary-state  system  whose 
components  fail  randomly  and  independently,  it  is  often  of 
interest  to  compute  the  probability  chat  two  terminals  or  nodes  s 
and  t  are  connected  and,  more  generally,  the  probability  that  all 
n  nodes  of  a  system  are  connected.  In  the  case  of  multi-state 
flow  systems  whose  components  have  randomly  determined  capacity 
flow  levels,  interest  often  focuses  on  the  distribution  of 
maximal  flow  from  a  source  node  s  to  a  terminal  node  t.  Both  the 
binary-state  connectivity  model  and  the  multi-state  flow  model 
can  be  used  to  represent  communication  and  transportation 
net wor  ks  . 

Since  computing  these  probabilities  belongs  to  the  class  of 
#P-hard  problems,  no  algorithms  are  known  to  solve  them  exactly 
in  time  polynomial  in  the  input.  As  an  alternative  method  of  solu¬ 
tion,  Monte  Carlo  sampling  plans  are  being  developed  to  estimate 
these  quantities,  especially  for  large  systems.  This  method  has 
the  advantage  that  errors  in  evaluation  are  essentially  indepen¬ 
dent  of  the  size  of  the  system.  This  paper  describes  how  these 
Monte  Carlo  sampling  plans  are  designed  to  use  prior  information 
on  the  system  under  study  to  obtain  estimates  with  specified  accu¬ 
racy  at  considerably  less  cost  than  crude  Monte  Carlo  sampling 
would  require.  Such  plans  are  called  variance  reducing  techni¬ 
ques  and  represent  an  integral  component  of  the  Monte  Carlo 
method.  Also,  thi3  prior  information  about  system  design  enables 
one  to  estimate,  prior  to  performing  any  sampling,  the  maximal 
sample  size  required  to  achieve  a  specified  accuracy  when  using 
the  variance  reducing  techniques.  These  worst  case  bounds  on 
sample  size  are  especially  helpful  when  a  limited  computing 
budget  is  available. 

The  paper  also  describes  how  Monte  Carlo  sampling  data 
accumulated  to  estimate  the  aforementioned  reliability  and  flow 
probabilities  can  be  used  to  perform  sensitivity  analyses  with 


r*. 


only  Incidental  additional  cost.  These  analyses  include  estimat¬ 
ing  the  effect  of  improving  component  reliability  in  the  binary- 
state  case  and  the  effect  of  increasing  the  probability  that  a 
component  operates  at  full  capacity  in  the  multi-state  flow  case. 

Virtually  all  computations  of  system  reliability  measures 
assume  that  component  reliability  probabilities,  which  are  the 
input  to  the  computation,  are  known  with  certainty.  This  is 
rarely  the  case  and  the  errors  that  do  exist  in  these  input  data 
directly  affect  the  quality  of  the  reliability  computation, 
whether  it  be  an  exact  or  a  Monte  Carlo  based  calculation.  This 
issue  is  discussed  in  detail.  In  particular,  the  paper  describes 
the  e r r o r - i n d u c e d  bias  and  variance  as  functions  of  the 
parameters  of  the  system  under  study  and  shows  how  these  measures 
can  adversely  affect  the  i  n t e r pr e t a t i o n  of  a  reliability 
calculation. 


Key  Words:  maximal  flow;  Monte  Carlo  sampling;  reliability; 
3-t  connectedness 
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Introduction 

-The  computation  of  system  reliability  from  component  reliabi¬ 
lities  presents  a  host  of  non-trivial  problems  for  systems  of 
varying  sizes.  These  include  the  functional  relationship  between 
the  time  required  to  compute  system  reliability  and  system  size. 
A  second  problem  concerns  how  system  reliability  varies  as 
component  reliabilities  vary.  A  third  problem  concerns  how 
statistical  errors  in  estimating  component  reliabilities  affect 
the  accuracy  of  the  system  reliability  computation. 

This  paper  describes  Monte  Carlo  techniques  which  provide 
useful  answers  to  the  first  two  problems  and  presents  an  analysis 
which  establishes  the  potential  seriousness  of  the  third  problem 
in  practice.  The  presentation  summarizes  research  reported  in 
[3.  5  ,  6,  7,  8,  9,  10,  11,  12].  To  put  the  results  in 
perspective,  we  begin  with  a  description  of  the  system 
reliability  concepts  that  are  of  interest  to  us. 

1  .  Characterizations  of  System  Reliability 

Consider  the  network  G  -  (V,E)  with  node  set  V  and  arc  set  E. 
For  convenience  of  exposition,  assume  that  nodes  represent  com¬ 
ponents  that  function  perfectly  and  that  arcs  represent  com¬ 
ponents  that  fail  randomly  and  independently.  To  characterize  G 
more  completely,  we  define: 


r  *  number  of  distinct  types  of  components 

F  •[  (  t  )  -  probability  that  a  component  of  type  i  functions 

at  time  x£0  for  i«1,...,r. 

F(t)  *  ( F ^ ( t ),..., F  ( x ) ) 

E^  *  set  of  arcs  that  use  components  of  type  i 

r 

(E.n  E.-0  i*j  E  -  U  E.  ) 

-1  -j  -  i-1  -1 

*  Ij-jJ  number  of  components  of  type  i 

k  -  (  k  ,  .  .  .  ,  k  ) 

-  I  r 

e  .  ,  *  1  th  arc  in  E . 

1  j  j  -1 

X .  j  *■  1  if  arc  e^  functions,  -  0  otherwise 
k . 

x.  -  ll  x.  .  -  number  of  arcs  of  type  i  that  function 


X  -  set  of  all  arc  states  x 

r  k  .  —  x  . 

P  (  x  ,  k  ,  F  (  x  )  )  -  H  [F.(  .  1  [1-Fi(t)]  1  1  xcX  (1) 

....  i  -1 

«  probability  mass  function  of  states  in  X  at  time  t 

<p(x)  -  1  if  the  system  functions,  -  0  otherwise  (2) 

g(F(t))  -  l  <p  ( x )  P(x,k,F(T))  -  probability  that  the  system 
xeX  ~  -  -  - 

functions  at  time  t.  (3) 

The  interpretation  of  the  system  reliability  g(F(t))  varies 
with  the  type  of  system  under  study.  For  a  binary-state  system 
with  binary-state  components,  we  consider  the  T-conneotednesa 
problem.  Let  T  denote  a  subset  of  V  and  let 

<(>  ( x )  -  1  if  all  nodes  in  T  are  connected  when  state  x  occurs 
-  0  otherwise. 

Then 

g ( F ( t ) )  -  probability  that  all  nodes  in  T  are  (4) 

connected  at  time  t  . 

When  T  »  { s  ,  t  }  c  V,  this  is  called  the  s-t  connectedness  problem. 

When  T  -  V,  it  is  called  the  all-terminal  connectedness  problem. 
These  representations  are  useful  in  studying  the  vulnerabi 1 ? t y  of 
communication  systems. 

One  can  also  use  the  network  representation  to  study  the 
reliability  of  multi-state  systems  with  binary-state  components. 
Suppose  G  is  a  directed  acyclic  flow  network  with  source  node  s 
and  terminal  node  t  and  that  at  time  t  each  component  of  type  i 
has  flow  capacities  zero  with  probability  1  —  F  ^ ( t  )  and  b^>0  with 
probability  F . ( t  )  .  Let 


-5- 

g(F(x))  at  all  tet  »  takes  0(k)  times  as  long  as  com¬ 

puting  g ( F ( x  )  )  at  a  single  point. 

To  overcome  this  limitation  to  exact  computation,  one  can 
resort  to  a  Monte  Carlo  sampling  experiment  wherein  one 
approximates  g(F(x))  by  an  estimate  whose  error  of  approxi mat i on 
decreases  as  the  number  of  independent  trials,  on  which  the 
estimate  is  based,  increases.  Many  alternative  sampling  plans 
exist  and,  in  principle,  one  prefers  a  plan  that  exploits  prior 
knowledge  about  the  system  of  interest  to  achieve  an  error  of 
specified  size  at  less  cost  than  alternative  methods  allow. 

We  now  describe  such  a  sampling  plan  that  applies  with  small 
modifications  to  the  estimation  of  diverse  reliability  measures 
such  as  (4),  (5),  (6),  (7)  and  (8),  when  they  are  appropriate.  For 
conciseness  of  exposition,  we  assume  that  estimation  is  to  be 
performed  for  fixed  F(x)  and  k  and  hereafter  suppress  these  argu¬ 
ments  unless  they  are  needed  to  avoid  ambiguity.  Reference  [4] 
provides  a  comparison  of  some  of  the  Monte  Carlo  techniques  to  be 
described  here  with  the  proposed  Monte  Carlo  sampling  plans  of 
other  writers. 

3 .  Using  Prior  Information 

Assume  that  sufficient  prior  information  exists  about 
the  system  under  study  to  enable  one  to  identify  two  binary 
functions  (<|>i(x)  xeX.}  and  { 2  (  x )  xeX}  with  the  property 

i|>1(x)S<|)(x)S(j>2(x)  (9) 

where  <t>  ( x )  is  defined  in  (2).  Let 

g,  -  l  $. ( x )  P(x) 
xeX 

so  that 

gj  S  g  S  g2 .  (10) 


Now  let 


which  is  a  probability  mass  function.  Suppose  that  one  can 
compute  g-j  and  g2  exactly  and  at  reasonable  cost  as  a  function  of 
the  size  of  G.  Then  the  ordering  relationships  (9)  and  (10) 
together  with  the  probability  mass  function  |q(x)  xeX}  in  (11) 
allow  one  to  benefit  from  two  alternative  modes  of  sampling. 

3*1  Importance  Sampling  [3.6,7,8,10] 

Here  one  concentrates  sampling  in  the  region  of  the  state 
space  X  1  -  1  xeX  |  4>  ^  (  x  ) -0  and  <J>2(x)*l}  as  follows: 

1  .  Set  S<-0. 

2.  On  each  of  K  independent  trials: 

2a.  Sample  x  from  {q(x)  xex). 

2b .  Compute  0 ( x  )  . 

2c  .  Set  S*S  +  <(>  (  x  )  . 

3.  Compute  summary  statistics 
3a.  gK  -  g1  +  (g2-g1 )S/K. 

3b.  V(gK)  ”  ( g2-gK gK"g1  )  ' (K” 1 ) ' 

Here  gK  is  an  unbiased  estimator  of  g  with 

var  gK  -  (g2~g) (g-g1 )/K  (12) 

A  A 

and  V(g  )  is  an  unbiased  estimator  of  var  g  . 

One  way  to  assess  the  benefit  of  this  sampling  plan  is  to 
compare  it  with  the  results  for  a  crude  Monte  Carlo  sampling 
estimate  g  of  g,  using  (p(x)}.  This  too  is  unbiased  but  with 

l\  _ 

var  gK»g(1-g)/K.  (13) 


The  ratio  of  variances  is  then 


R j-  -  var  gK/var  gK 

2  1/{g2-g1-2Cg1g2(1-g1  )  (l-g2)  (14) 

Z  1  . 


Since  the  lower  bound  (14)  is  computable  prior  to  sampling,  one 
can  obtain  a  clear  understanding  of  the  least  favorable  result 
that  the  sampling  plan  can  induce  before  employing  it. 

The  variance  ratio  merely  measures  one  aspect  of  the  impor¬ 
tance  sampling  plan.  Let  c(P)  and  c(Q)  denote  the  mean  times  per 
replication  of  employing  sampling  plans  based  on  { P ( x ) }  in  (1) 
and  { Q ( x ) }  in  (11)  respectively.  Note  that  each  of  these  includes 
the  cost  of  computing  <f >(x).  Then 


W 


I 


c(P)  R 
c(Q)  HI 


denotes  the  mean  time  required  to  achieve  the  same  variance  with 
crude  Monte  Carlo  sampling  as  one  obtains  in  one  unit  of  time 
with  the  importance  sampling  plan.  For  importance  sampling  to  be 
beneficial,  one  requires  Wj>1,  preferably  considerably  greater 
than  1.  Reference  [3]  describes  how  to  construct  {q(x)}  and  how  to 
sample  from  it  for  T-connectivity,  and  [10]  describes  the 
corresponding  construction  for  the  reliability  measure  g(F(t),z) 
in  (5).  In  both  cases,  the  ability  to  determine  sets  of  edge- 
disjoint  minimal  spanning  trees  and  minimal  cutsets  enables  one 
to  identify  the  binary  functions  j  4>  1  ( x )  }  and  $2(x)}  needed  to 
achieve  the  variance  reduction.  References  [3]  amd  [10]  also 
describe  how  to  compute  confidence  intervals  for  g  that  hold  for 
finite  K.  These  considerably  improve  on  normal  confidence 


intervals  whose  use  inevitably  introduces  an  additional  error  of 
approximati on  . 


3.2  Control  Variates  [9] 

This  plan  uses  inequalities  (9)  and  (10),  but  samples  the 
state  vector  x  from  { P ( x )  }  in  (1)  instead  of  from  { Q  ( x )  }  in  (11). 
In  particular: 

1  .  Set  S<-0 ,  S i  0  and  S2*'0. 

2.  On  each  of  K  trials: 

2a.  Sample  x  from  { P ( x  )  }  . 

2b.  Compute  ^(x),  $(x)  and  $2(x). 

2c.  Set  S1  ♦S1  +<)>  (  x  )  ,  S«-S  +  $(x)  and  S  2<-S2  + 4>2  (  x  )  . 

3-  Compute  summary  statistic 


S(g2-g.)+(S/K-g)(S-Kg)+(g-S/K)(S-Kg)-g1 


The  resulting  statistic  is  an  unbiased  estimator  of  g  with 

lim  K  var  gK  -  ( g 2~ g ) ( g- g 1  ) / ( g 2~ g 1  )  .  (16) 

K  -*•« 

An  unbiased  estimator  of  var  gv  is  also  given  in  [10]. 

ft 

Observe  that  for  large  K 

var  gK  =•  var  gK/(g2"g1)  >  var  SK-  (17) 

Although  sampling  from  { Q ( x ) }  for  g  consumes  more  time  than  samp- 

—  ft 

ling  from  {p(x)|  for  g  does,  experience  has  shown  that  importance 
_  ft 

sampling  tends  to  be  more  statistically  efficient  than  the  method 
of  control  variates  is  for  reliability  estimation  at  a  single 
point.  However,  in  the  network  flow  model  the  control  variate 
approach  produces  sample  data  that  allow  estimation  at  more  than 
one  flow  point  z  whereas  importance  sampling  does  not.  In  parti¬ 
cular,  [10]  shows  how  this  can  be  done  in  estimating  the  comple¬ 
mentary  distribution  of  maximal  s-t  flow  {g(F(t),z)}  at  a  set  of 
points  zeZ.  Confidence  intervals  for  }g(F(x),z)  zcz)  are  also 
gi ven  there . 
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4 .  Determining  the  Worst  Case  Sample  Size 

In  addition  to  producing  a  statistically  more  efficient 

estimator  than  crude  Monte  Carlo  sampling  does,  an  effective 

Monte  Carlo  technique  also  provides  information  that  enables  one 

to  determine  how  large  a  sample  size  is  necessary  to  achieve  an 

estimate  with  a  specified  upper  bound  on  statistical  error.  Both 

the  importance  sampling  and  control  variate  approaches  provide 

such  a  result  based  on  a  worst  case  scenario. 

Let  X X„  denote  i.i.d.  random  variables  with  u  =  EX.  and 
I  K  1 

pr(a^Xi^b)  =*  1  for  all  i  =  1 . K.  Then  for  X  =  (u~a)/(b-a)  and 

e <mi n ( A , 1  -  X )  ,  Hoeffding  (  1  963  )  shows  that 

pr(XK-u2e)  <  RK(  A  ,  e/ (b-a)  ) 


and 


pr (-XK  +  y>e/ (b-a)  )  <  R  K  (  1  -  A  ,  e  /  (  b- a  )  ) 

where 

R  (  A  ,  0  )  -  (  yTq)  X+Q  ( ~ -  0  ) 1  ~  *  "  9  0<9<min(  X  ,  1-A  )  ,  0  <  X  <  1  . 

Note  that  R  has  a  unique  minimum  for  X  in  [  0  ,  1  -  e /  (  b- a  )  ]  . 

Suppose  that  one  wants  to  determine  the  minimal  sample  size 
K*  such  that 

Pr ( | XK„-u  |  Sc  )  2  1  -  a  (18) 

for  sampling  absolute  accuracy  e  and  confidence  level  1  -  a.  Then 
collecting 

K  »  ln(o/2)/max  In  R(A,e/(b-a))  (19) 

0SAS1 

observations  assures  the  absolute  error  bounds  in  (18). 

Now  suppose  that  y > 0  and  consider  the  relative  error 


criterion 


mvy. 


I* |I  lla'I 


L*J 
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pr  (  |  X^-y  |  Sey  )  >  1  -  a.  (20 

Then  K*  is  again  computable  as  in  (19)  with  e  y  replacing  e  so  that 

K  =  ln(a/2)/min[  max  In  R  (  X  ,  +X  )c  ) ,  max  In  R  ( 1  -  X  ,  (z~— +  1  )  e  )  ] .  (21 

0SX<1  b_a  0<XS1  b_a 

For  the  reliability  problem  using  importance  sampling  with 
XK  ■  8K »  a  =  8l  and  b  ■  g  2 »  the  significance  of  (19)  and  (21)  lies 
in  the  realization  that  one  can  simply  take  these  numbers  of 
observations  and  be  guaranteed  the  corresponding  specified 
accuracy  without  any  subsequent  analysis  after  computation  of  g 

K 

A  worst  case  analysis  also  exists  for  the  control  variate 
sampling  plan.  Let 

r  (  X  ,  e  ,  h  ,  g 1  ,gj)  =  e  hE{l-g2  +  g1+(g2-g1)[Xe(1  A)h+(1-X)e  Ah]} 


g2~g1 


h£  0  0<  e  £  m  i  n  (  X  ,  1  -  X  )  . 


To  ensure  an  absolute  accuracy  e  with  probability  2  1  -  a,  one 
needs  no  more  than 

K  =  ln(a/2)/min{  max  [In  min  R ( X , h ,g  ,g_) ] ,  max  [In  min  R ( 1 - X , h ,g  ,g? )  ]  )  (23) 
0<X<1  n>0  0<X£1  h>0 

observations.  In  spite  of  its  formidable  appearance,  one  can 
evaluate  (23)  using  a  modification  of  algorithm  A  in  [9]. 

5 .  Sensitivity  [12] 

In  addition  to  estimating  the  reliability  g  ( F ( t  )  )  at  a  fixed 
time  t,  the  Monte  Carlo  method  provides  a  way,  at  small  marginal 
cost,  of  estimating  the  more  general  reliability  function 
{  g  (  q  )  q  -  ( q i  ,  .  .  .  ,  qr  )  e W  }  where  q.  =  probability  that  a  component  of 
type  i  functions  i«1,...,r  and  W  is  a  set  of  component  reliability 
vectors  of  interest.  Here  q^  for  component  i  may  be  a  function  of 
time  (i.e.  qi=F^(x))  o_r  it  may  reflect  potential  component  relia¬ 
bility  improvements  the  effect  of  which  on  system  reliability  is 
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of  interest. 

With  this  change  of  nomenclature,  one  has 

P(x,k,q)  -  n  q  1 (1-q. )  1  1  (24) 

-  -  -  i-1 

and 

g  ( q )  »  l  <J»(x)  P(x,k,q)  qeW.  (25) 

x  e  X  ~  ~  ~  ~ 

Suppose  that  one  elects  to  perform  K  independent  replications 
with  component  reliability  p  using  {Q(x,k,p)}  (importance  sampl¬ 
ing).  Let  x(j)  denote  the  sample  component  state  vector  on  repli¬ 
cation  j.  The  estimators 

K  P(x(J),k,q) 

g1K.(q)  =*  g.(q)  +  [g?(p)-g.  (p)]K  1  I  4>(x(j))  - 7TT -  (26) 

1K  -  1  ~  2  -  j-1  "  P(xU\k,p) 

and 

K  P(x(J),k,q) 

g  (q)  =«  g  (q)  -  [g?(p)~g.  (p)]K_1  l  [1-4>(x(j))]  - rr-c -  (27) 

2K  -  2  ~  2  ~  1  ~  j  =  1  -  P(xU\k,p) 

are  unbiased  estimators  of  g(q)  with 

Kv1 (q)  -  K  var  g1[((q)  -  {c[g2(p)-g., (p) ][g(q  )-g1  (q  )  D'CgtqJ-g,  (q )  ][g2(q  )“g,  (q )  ] } 

+  [g2(q)-g(q)][g(q)-g1 (q)3  (28) 

Kv^Cq)  »  K  var  g2[<(q)  =  {c[g2(p)-g1  (p)][g2(q  )-g(q  )  >[g2(q)~g(q)  3Cg2(q)~g1  (q)  ]f 

+  Cg2(q)-g(q) ][g(q)“g1 (q) ]  (29) 

Kc12(q)  -  K  C0VCS1K(q)-g2K(q)]  “  Cg2(q)-8(q)]Cg(q)-g1(q)]  (30) 
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wher  e 

r  k 

c  -  n  [qf/p . +( 1 -q. ) 2/ ( 1 -p  )  ]  1, 
i-1  1  1  1  1 

*  *  * 

q  -  (q1 . qp) 

and 

*  2 

q.-q./cp.  i-1 . r. 

xii 

In  principle,  (26)  and  (27)  enable  one  to  estimate  g(q)  for 

all  qeW  from  the  single  set  of  replications  obtained  by  sampling 

with  component  reliabilities  p.  Since  [ g2 ( q ) - g ( q )  ][ g ( q ) - g 1 ( q  )  ] 

in  (28)  and  (29)  is  the  variance  of  the  point  estimator  g^Cq) 

based  on  sampling  with  component  r el i abi 1 i t i es  q,  the  quantities 

in  curly  brackets  are  the  incremental  changes  in  variances  that 

result  from  sampling  with  p  instead  of  q.  Of  most  importance,  it 

is  entirely  possible  for  one  of  these  quantities  to  be  negative, 

implying  that  an  estimate  of  g(q)  with  smaller  variance  than  g  ( q  ) 

~  K 

is  possible. 

To  put  this  last  observation  in  perspective,  consider  the 
estimator 


iK(q)  -  0g1K(q)  ♦  (1~0)g2K(q) 


O<0S1 .  ( 31  ) 


which  achieves  minimal  variance  by  choosing 


0 


1  if 

v2(q)-Ci2<q> 
v  1  (  q  )  +v2(  q  )  -  2c  1  2  (  q  )" 


v1 (q)Sc12(q) 

v 1 (q ) , v2( q ) ic1 2 ( q  )  (32) 


-  0 


v  2 ( q ) S  c 1 2(9>  • 


Observe  that  v^qJSc^Cq)  implies  that  g 
variance  than  gK(q)  in  (12)  whereas 

a 

g2K(q)  in  (27)  has  smaller  variance  than 


1K(q)  in  (26)  has  smaller 
v2(q)Sc12(q)  implies  that 
gK(q)  in  (12). 


Experience  with  this  method  of  estimation  indicates  that  for 
moderately  high  component  reliabilities  Vg(q)Sc12(q)  is  often 
satisfied  with  substantial  reductions  in  variance  at  each  q  in  W, 
as  compared  to  the  results  from  importance  sampling  at  a  point  in 
Section  3  •  1  •  Moreover,  all  these  estimates  follow  from  just  one 
sampling  experiment  at  p.  The  development  of  a  confidence  region 
for  the  reliabilty  function  { g ( q )  qew}  is  now  underway,  as  is  a 
study  to  determine  the  optimal  p  at  which  to  sample  from  the  com¬ 
ponent  state  space  X. 

6 .  Errors  in  Input  Parameters  [11] 

Regardless  of  whether  an  exact  or  Monte  Carlo  method  is  used 
in  computing  system  reliability  g(q),  it  is  customary  to  assume 
that  the  numerical  values  of  the  component  reliabilities 

q  -  (q1 . q^)  are  exact.  In  reality  they  are  not,  and,  as  we  now 

show  that  ignoring  this  potential  source  of  error  can  give  a 
misleading  interpretation  to  the  final  numerical  value  computed 
for  g ( q )  . 

Suppose  one  tests  n^  components  of  type  i  for  i-1 . r. 

Each  test  begins  with  a  new  component  functioning.  Let  Z^j 

denote  the  outcome  of  the  jth  test  of  component  of  type  i  where 

Z ^ j  ■=  1  if  the  component  functions  at  the  end  of  the  test  period 

and  Z^j  -  0  if  the  component  fails  prior  to  the  end  of  the  test 

period.  Presumably  each  component  of  type  i  is  tested  under 

identical  conditions  that  resemble  the  system  environment.  Then 

one  has  the  data  vectors  Z.  -  { Z Z .  }  for  i»1,...,r  where 

l 1  in, 

l 

the  elements  of  Z^  are  independent  and  identically  distributed 
with  q^  «  EZij  j*1,...,n^  and  Z ^ , . . . , Z^  are  independent.  Also, 


is  the  maximum  likelihood  estimator  of  q^  with 


qi C 1 ■ qi )x K 


var  q^^  - 

E(qi-qi)k  -  0(1/niL(k+1 )/2J  )  k>2 


as 


n 


i+“* 


A  A  ^ 

Let  q  -  (q^,...,q  ).  Then  it  is  not  unusual  to  compute 
system  reliability  as  g(q)  with  no  mention  of  the  sampling  error 
that  replacing  q  by  q  induces.  Unfortunately,  the  results  in 
Fishman  (1987)  show  that 

Eg(q)  -  g(q)  ~  w(k,q,n)  as  min  n  -*•»  (34 

-  -  -  1 SiSr 

and 

var  g ( q )  „  v(k,q,n)  a3  min  n  ♦»  (35 

-  -  -  1 SiSr  1 

where 

n  -  ( n  ^  ,  .  .  . , n^  ) , 


r  k(k-1)q-2x(k-1)q+x(x-1)  q  ( 1-q  ) 

w(k,q,n)  -  l  {  l  *(x)P(x,k,q)[-i-i - 1..  A..1.  1  1  1 - ] }  _i - L.  (36 

xeX  xeX  -  -  -  qi(1“qi'  2n.. 


and 


v(k,q,n)  -1(1  <()(x)P(x,k,q)  [ 


Vkiqi 


i-1  xeX 


qjO-qj) 


,)2V^V 


(37 


Expressions  (3*1)  and  (35)  imply  that  even  in  the  case  of  an  exact 
computation  (no  Monte  Carlo  sampling),  a  substantial  degree  of 
statistical  error  can  arise.  Clearly  g(q)  is  biased.  Moreover, 
this  bias  (36)  and  the  variance  (37)  grow  q ua dr  a t i c a  1  1  y  with 
k  1  ,  .  .  .  , kr  the  number  of  components  of  each  type  used  in  the 
system  and  linearly  with  the  number  r  of  different  types  of 
com ponents  . 


Of  special  interest  is  the  observation  that  if  all 
components  of  type  i  are  in  series  and  q^  replaces  q^  in  the 
reliability  computation,  then  the  resulting  reliability 
overstates  the  true  reliability.  Conversely,  an  understatement 
of  reliability  occurs  when  all  components  of  type  i  are  in 


parallel  . 


^vnr»v»i'»v"j?Tvnr»: 
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Since  computing  these  probabilities  belongs  to  the  class  of 
#P-hard  problems,  no  algorithms  are  known  to  solve  them  exactly  in 
time  polynomial  in  the  input.  As  an  alternative  method  of  solu¬ 
tion,  Monte  Carlo  sampling  plans  are  being  developed  to  estimate 
these  quantities,  especially  for  large  systems.  This  method  has 
the  advantage  that  errors  in  evaluation  are  essentially  indepen¬ 
dent  of  the  size  of  the  system.  This  paper  describes  how  these 
Monte  Carlo  sampling  plans  are  designed  to  use  prior  information 
on  the  system  under  study  to  obtain  estimates  with  specified  accu¬ 
racy  at  considerably  less  cost  than  crude  Monte  Carlo  sampling 
would  require.  Such  plans  are  called  variance  reducing  techniques 
and  represent  an  integral  component  of  the  Monte  Carlo  method. 
Also,  this  prior  information  about  system  design  enables  one  to 
estimate,  prior  to  performing  any  sampling,  the  maximal  sample 
size  required  to  achieve  a  specified  accuracy  when  using  the 
variance  reducing  techniques.  These  worst  case  bounds  on  sample 
size  are  especially  helpful  when  a  limited  computing  budget  is 
available. 

The  paper  also  describes  how  Monte  Carlo  sampling  data  accumu¬ 
lated  to  estimate  the  aforementioned  reliability  and  flow  probabi¬ 
lities  can  be  used  to  perform  sensitivity  analyses  with  only  inci¬ 
dental  additional  cost.  These  analyses  include  estimating  the 
effect  of  improving  component  reliability  in  the  binarystate  case 
and  the  effect  of  increasing  the  probability  that  a  component 
operates  at  full  capacity  in  the  multi-state  flow  case. 

Virtually  all  computations  of  system  reliability  measures 
assume  that  component  reliability  probabilities,  which  are  the 
input  to  the  computation,  are  known  with  certainty.  This  is 
rarely  the  case  and  the  errors  that  do  exist  in  these  input  data 
directly  affect  the  quality  of  the  reliability  computation, 
whether  it  be  an  exact  or  a  Monte  Carlo  based  calculation.  This 
issue  is  discussed  in  detail.  In  particular,  the  paper  describes 
‘he  er r or- i nd uced  bias  and  variance  as  functions  of  the  parameters 
'  the  system  under  study  and  shows  how  these  measures  can  adver- 
aly  affect  the  interpretation  of  a  reliability  calculation. 
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